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Abstract 

In this paper, new classes of lower bounds on the outage error probability and on the mean-square-error (MSE) in 
Bayesian parameter estimation are proposed. The minima of the ^-outage error probability and the MSE are obtained by 
the generalized maximum a-posteriori probability and the minimum MSE (MMSE) estimators, respectively. However, 



J^* , computation of these estimators and their corresponding performance is usually not tractable and thus, lower bounds 



on these terms can be very useful for performance analysis. The proposed class of lower bounds on the outage error 



p; 

iy-T , probability is derived using Holder's inequality. This class is utilized to derive a new class of Bayesian MSE bounds. It 

^^ , is shown that for unimodal symmetric conditional probability density functions (pdf ) the tightest probability of outage 

^H . 

^ , error lower bound in the proposed class attains the minimum probability of outage error and the tightest MSE bound 

•i— I , 

rS | coincides with the MMSE performance. In addition, it is shown that the proposed MSE bounds are always tighter than 

a ; 

the Ziv-Zakai lower bound (ZZLB). The proposed bounds are compared with other existing performance lower bounds 
via some examples. 
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I. Introduction 

The mean-square-error (MSE) criterion has been commonly used for performance analysis in parameter 
estimation. Lower bounds on the MSE are widely used for problems where the exact minimum MSE (MMSE) 



is difficult to evaluate. Bayesian MSE bounds for random parameters estimation can be divided into two 
classes. The Weiss- Weinstein class is based on the covariance inequality which includes the Bayesian CRB 
(BCRB) [1], the Reuven-Messer bound [2], the Weiss- Weinstein lower bound (WWLB), and the Bayesian 
Todros-Tabrikian bound [3]. The Ziv-Zakai class of bounds relates the MSE in the estimation problem to the 
probability of error in a binary detection problem. The Ziv-Zakai class includes the Ziv-Zakai lower bound 
(ZZLB) [4] and its improvements, notably the Bellini- Tartara bound [5], the Chazan-Zakai-Ziv bound [6], and 
Bell- Ziv-Zakai bound [7]. 

Additional important criterion for performance analysis in parameter estimation is the probability of outage 
error, which is the probability that the estimation error is higher than a given threshold. This criterion 
provides meaningful information in the presence of large errors, while the occurrence of large errors with small 
probability may cause the MSE criterion to be non-informative. In some parameter estimation problems we 
may be interested in evaluation of outage error rate and the exact value of the error may be non-informative [8] , 
[9] . The probability of outage error criterion as a function of threshold error provides information on the error 
distribution while the MSE provides information only on the second order moment of the estimation error. 
In addition, in many estimation problems the MSE is subject to a threshold phenomenon which determines 
operation region (see papers in [10]). Thus, the MSE threshold may be highly influenced by large errors with 
small probability and thus the outage error probability criterion can be more useful for this propose. 

The minimum outage error probability can be obtained by the generalized maximum a-posteriori probability 
(MAP) estimator which is given by maximization of the posterior function convolved with a rectangular 
window. However, computation of the minimum outage error probability is usually not tractable, and thus 
tight lower bounds on the probability of outage error are useful for performance analysis and system design. 
In the literature, only few lower bounds on the probability of outage error can be found and most of them are 
based on the probability of error in binary or multiple hypothesis testing problems. A known lower bound 
for uniformly distributed unknown parameters is given by the Kotel'nikov's inequality [11]. There are several 
works on approximations of the probability of outliers for non-Bayesian direction-of-arrival (DOA) estimation 
problem (see e.g. [12], [13]). In general, the internal terms in the integral version of the ZZLB can be used as 



lower bounds on the outage error probability. In similar, MSE lower bounds can be obtained from outage error 
lower bounds by using the Chebyshev's inequality, as in the original ZZLB [4], or by using the probability 
identity [14], as in [5], [7]. The Chebyshev's inequality is known to be unachievable and thus the second 
option is preferred. The outage error probability can be interpreted as the probability of error for estimation 
problems. General classes of lower bounds for the probability of error in multiple hypothesis problems have 
been derived in [15]. 

In this paper, a new class of Bayesian lower bounds on the minimum probability of outage error is derived 
using Holder's inequality. In some cases, the proposed outage error probability bounds are simpler to compute 
than the minimum outage error probability and they provide a good prediction of this criterion. It is shown 
that for parameter estimation problems with unimodal symmetric conditional probability density function 
(pdf), the tightest lower bound under this class coincides with the optimum probability of outage error 
provided by the generalized MAP criterion. In addition, using the probability identity [14] new classes of 
Bayesian lower bounds on arbitrary distortion measures are derived. These classes are based on the minimum 
outage error probability bounds, derived in the first part of this paper, and on the probability identity [14]. 
For the new class of MSE lower bounds, it is shown that the tightest bound in this class is always tighter than 
the ZZLB. For parameter estimation problems with unimodal symmetric conditional pdf, the tightest bound 
in the proposed class attains the MMSE. 

The paper is organized as follows. The problem statement is presented in Section II. In Section III, the 
new class of Bayesian lower bounds on the probability of outage error is derived and in Section IV the tightest 
subclass of lower bounds in this class is found. A new class of lower bounds on different distortion measures 
is derived in Section V using the bounds on the probability of outage error. In particular, a new class of MSE 
Bayesian lower bounds is derived. The bounds properties are described in Section VI, and the performance 
of the proposed bounds are evaluated in Section VII via some examples. Finally, our conclusions appear in 
Section VIII. 



II. Problem statement 



Consider the estimation of a continuous scalar random variable 9 € M, based on a random observation 
vector x € x with the cumulative distribution function (cdf) F x (-) and Fg| x (-|x) denotes the conditional 
cdf of 9 given x. It is assumed that i^| x (-|x) is continuous such that the conditional pdf exists and that 
/gi x ((/?|x) > for almost all (p £ R and x £ \- For any estimator, #(x) : x — >■ R, the estimation error 



is #(x) — 9 and the corresponding /i-outage error probability and MSE are given by Pr 
E 9 — 9 , respectively. The minimum probability of /i-outage error is (see e.g. [16]) 
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min Pr 

0(x) 
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max / f e \ x (<p\x) dtp 
0(x) J 

0(X)-| 



which is attained by the following estimator 



9 h (x) = argmax / f e \ x (ip\x) dtp 

e(x) Je(x)-h 



(2) 



The MAP estimator is obtained by (2) in the limit h — > 0. Thus, the estimator in (2), named /i-MAP, 
is a generalized version of the MAP estimator for any threshold h. This estimator can be implemented 
by maximization of the conditional pdf, /6i| x ('l x ) after convolution with an ft,-width rectangular window. 
Calculation of the minimum /i-outage error probability in (1) as well as the MMSE is usually not tractable. 
Tight lower bounds on these performance measures are useful for performance analysis and system design. 
Lower bounds on the outage error probability can be useful also for derivation of MSE bounds, as will be 
demonstrated in Section V. In addition, the outage error probability lower bounds can be useful for upper 
bounding the cdf of the absolute error, given by 
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(3) 



III. A GENERAL CLASS OF LOWER BOUNDS ON /l-OUTAGE ERROR PROBABILITY 



A. Derivation of the general class of bounds 



Let Uh(x-,0) denote an indicator function 
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(4) 



where 0(x) is an estimator of the random parameter e. According to reverse Holder's inequality [17] 



E[Mx,0H(x,0)|]>EP \u h (x,6)\r E 1 -? \v h (x,e)\— P 



\/p> 1 



(5) 



for any arbitrary scalar function Vh(x,9) such that Vh(x,9) ^ for almost all 9 G R, x G x an d subject to 
the existence of these expectations. Using (4), one obtains 
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(6) 



(7) 



By substitution of (6) and (7) into (5) one obtains the following lower bound on the outage error probability: 



Pr 



>_] >i_Ep 
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E p 



\v h {x,0)\i-p 



(8) 



In general, this bound is a function of the estimator e. The following theorem states the condition to obtain 
valid bounds which are independent of the estimator. 

Theorem 1: Under the assumption that /6i| x (^l x ) > f° r almost all G R, x € Xi a necessary and sufficient 
condition for the lower bound in (8) to be a valid lower bound which is independent of the estimator e, is 
that the function 



A 



0,(x,0) = / fl | X (0|x)MM)| 



(9) 



is periodic in e with period /i, for a.e. x G x- 
Proof: The proof appears in Appendix A. 



The periodic function gh(x.,0) is chosen such that it is also piecewise continuous with respect to (w.r.t.) 8, 
and has left and right-hand derivatives V# £ [0, h]. Thus, <?/i(x, 6) can be represented using Fourier series [18]: 



9h(x,9)= ^2 a k {x,h)e % l e , a .e. x E x 



(10) 



k=— oo 



Using (6), (7), (9), and (10) we obtain 
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By substituting (12) and (11), the bound in (8) can be rewritten as 



J2 a k (x,h)e iJ t^d(p 



/iE [ao(x, 



(12) 
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where 



£ A =l-fepEp[a (x,/i)]E E F i 

2 'P 



£ afcCx.fcJe*^! /^( V |x)d V 
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<*h) = j- J2 / a k (x,h)e i2 ^d<p = j- Y, ak(x,h)e i2 -¥*d<p. 



(14) 



(15) 



fc=-oo ,/u " u fc=-oo 

Using different series of functions {afe(x, h)} and p, one obtains different bounds from this class. It should 
be noted that according to (10), {afc(x, h)} should be chosen such that gh(x,8) is a positive function and in 
particular, ao(x, h) > 0. In addition, the series {afc(x, h)} should be in ^(^) where ^(R) is the Hilbert space 
of square-summable real sequences, such that the Fourier series converges. 

> 2 ) is a non-negative non-increasing function of h, the class of bounds in (14) can be 



Since Pr 

improved using the "valley-filling" operator [5], [7]. The "valley-filling" operator, V, returns a non-increasing 
function by filling in any valleys in the input function, i.e. for function /(•) the operator results 



V f(h) = max f(h + £), h>0. 



(16) 



In addition, using the non-negativity property of the probability of outage error, negative values of the bound 
are limited to zero. 



B. Example - single coefficient bounds 

In this section, an example for derivation of bounds from the proposed class by choosing specific Fourier 
coefficients is given. By substituting the choice ao(x, h) > and ak(x-,h) = 0, VA; ^ in (14), one obtains 
the class of single coefficient bounds 
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Now, we maximize the bound w.r.t. ao(x, h). According to Holder's inequality [17] 
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, p>l, a (x,/i) >0 (18) 



for all non- negative functions ao(x, h) and it becomes an equality iff 



a (x,/i) =c( / f*\ x (<p\x) dtp 



£-1 

P 



(19) 



where c is a positive constant. Thus, by substituting (19) into (17), one obtains the following tightest single 
coefficient bound: 
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(20) 



Since the probability of error should be non-negative, the bound in (20) can be modified to 
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?(i) 



> - > max <^ 0, B\ ' } = max { 0, 1 - h?E 
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IV. The tightest subclass of lower bounds on the probability of outage error 



According to Holder's inequality [17] 
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which becomes an equality iff 

a (x,h)=c(h) I ( ^ a k (x,h)en 



Vp > 1 (22) 



i 
i-p _g_ 



'0|x 



(23) 



\fc=— oo 



where c(h) denotes a positive constant independent of x and 9. Thus, for given coefficients {afc(x, h)} , k ^ 
the tightest subclasses of bounds in the proposed class is 



B hn = l-hpE 



«0 ( x ' h ) 
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1-p 
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(24) 



Let assume that the Fourier series of the functions Ph(x, 6) and g' h (x,9) = gg ' w.r.t. 9 converges 
uniformly for almost all 9 £ R, x € \. Under this assumption, the bound in (24) can be maximized w.r.t. 
{afc(x, h)} , k 7^ by equating its corresponding derivatives to zero. Under the assumption that the integration 
and derivatives can be reordered, one obtains 



J [ E 4 0) ( x ^K^H /Jj x 1 Mx)e 4 ^^d v3 = C(x,/ l )5H, Vm£Z,xe X 



(25) 



\k=— oo 



fcG2 



where Z is the set of integers, <£[•] denotes the Kronecker delta function, and the Fourier coefficients s ajf > 
are the coefficients that maximize the bound in (24). In Appendix B, it is shown that the stationary point 
satisfying (25) yields a maximum of the bound in (24). Under the assumption that for every ip there is 
loi'-p) £ Z such that 



/e|x(</> + ^l x ) < 



1 



(l+a) 



p-1 ' 



V^eK, |/| > \l (<p)\, Z eZ, 



(26) 



for arbitrary a > 0, the series Yl'bL-oo f^ 1 (^ ~^~ ^l x ) converges for given p and the integral in the l.h.s. of 
(25) can be divided into an infinite sum of integrals, where each integral is over a single period, h, and the 
delta function can be replaced by its Fourier series representation on [0, h]. Thus, (25) can be rewritten as 
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(27) 



V?n 6 Z, x € x. Using the uniqueness of the Fourier series representation for continuous functions [18], 



l-jj oo 
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for almost all € M, x £ x? an d thus the function gh(x,(p) from (10) that maximizes the bound in (24) can 



be expressed as 
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gf(x,0)= £ 4°V>K^ = (5(x,/0 J2 fe P \x(8 + lh\x) 
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where C(x,h) = I ^' - I P . By substituting (28) in (15), one obtains 
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/i / °° _p_ 

C(x,fc) E /^(V + ^W) dp. (29) 

V=-oo / 



k=— oo 

By substituting (28) and (29) in (24), the tightest subclass of bounds in this class is given by 



5l o) = 1 - E 



p-i 
p 



E /^(v+^w d v 



\l=— oo 



, Vp> 1 



(30) 



p-i 
p 



The term I Yl'iZ-oo fe\x i'P + ^l x ) ) * s the -^j- norm of {fe\ x (^p + //i|x)} , and if the assumption in 
(26) satisfies for all p > 1, it converges for p > 1. Since the -^j norm is an increasing function of p [17], the 
class of bounds in (30) satisfies 

(31) 



s! o) >4 o) VKp<r 

O'P ■}. '" 



and 



sl o) > lim S ( h o) =0, p > 1 



2 ,P p->00 2>p 

and thus, the bound is non-negative Vp > 1. In particular, for p that approaches 1 from above, i.e. p — > 1 
the bound in (30) becomes 



(32) 



£ 



(°) 



1-E 



h 



m ax{/ | x (v? + //i|x)} dp 



(33) 



which is the tightest bound on the outage error probability in the proposed class of lower bounds. 

The derivations of the proposed bounds in Sections III and IV are carried out under the assumption that 
fe\ x (tp\x) > for almost all p € M. and x € x- Extension to any conditional pdf, i.e. /e| x (¥'l x ) — f° r an 
ip € M. and x E x> i s performed in Appendix C. 
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V. General classes of lower bounds on arbitrary distortion measures and MSE 

A. Derivation of the proposed class of bounds 

In similar to the extension of the ZZLB to arbitrary distortion measures [19], the proposed outage error 
lower bounds in (14) and (30) can be used to derive lower bounds on any non-decreasing (for non-negative 
values) and differentiable distortion measure D(-) with -D(O) = and derivative D(-). The expectation over 
the distortion measure is 



E 



D 



1 



x D(i\ 



>\\ 



(34) 



For non-decreasing JD(-) where D(-) is non- negative for positive arguments, (34) can be lower bounded by 



bounding the probability of outage error, Pr 



average distortion measures is obtained by substituting Bh from (14) in (34): 
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For example, for D 
rewritten as [14] 
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, n G N where N is the positive integers set, (34) and (35) can be 
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>ir Bhh n - L dh, neN,p>l, 



(37) 



respectively. Thus, the proposed lower bound on outage error probability can be used to bound any moment 
of the absolute error in Bayesian parameter estimation. In particular, a new class of MSE lower bounds can 
be obtained by 



E 
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Pr 



> \ ' 
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and 
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(39) 
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In order to obtain tighter MSE bounds, one should use tighter outage error probability lower bounds. Thus, 
by substituting the tightest subclass of lower bounds on the outage error probability from (30) in (39), one 

2" 



obtains E 



> Cp° where 



c v ] = \l B t hdh= \l 1_E /(£/*ii> + fcw) : d ^ 



£-1 

V 



\l=—oo 



hdh, 



(40) 



which is the tightest subclass of MSE bounds for a given p. By substituting the tightest outage error probability 
bound, B° from (33) into (40), one obtains 



c 



(o) A 1 



1-E 



max/e| x (^ + //i|x)dv3 



hdh 



(41) 



which is the tightest MSE bound in this class. 

B. Example - single coefficient MSE bounds 

In Section III-B, the single coefficient outage error probability is described. By substituting (21) in (39), 
one obtains the following MSE bound 

Ap 
1 f (. ,1 . \ , ,, 1 



E 



> 



1 - hpA p ) hdh 



(42) 



where A p = E 



tractable than other existing bounds. 



£-1 

p 



4(2p + 1)A 2 P P 
. Although this bound is not tight in the general case, it may be more 



VI. Properties of the bounds 

A. Relation to the ZZLB 

Theorem 2: The proposed MSE bound in (41) is always tighter than the ZZLB [7] for /<j| x (0|x) > 0, V6 1 € K 
and for almost every x G x- 
Proof: Appendix D. 

B. Tightness 

Theorem 3: If the conditional pdf /e| x ('l x ) is unimodal, then the outage error probability bound in (33) 
coincides with the minimum outage error probability in (1) for every h > 0. 
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Proof: Appendix E. 

In addition, it was shown by Bell [19] that the ZZLB in (79) coincides with the minimum MSE when the 
conditional pdf is symmetric and unimodal. According to Theorem 2, the proposed MSE bound in (41) is 
always tighter (or equal) than the ZZLB. Thus, if the conditional pdf is symmetric and unimodal, the MSE 
bound in (41) coincides with the minimum MSE. 

C. Dependence on the likelihood ratio function 

Let L X) e(xo,y + lh,(p) = '? ?^ ■, be the joint likelihood-ratio (LR) function of the pdf of x and 9, 
/x,6»('>')> between the points (x, 6) = (xo, <p + IK) and (x, 6) = (xo,</?). Accordingly, the bounds on the 
probability of outage error in (30) and (33) can be rewritten as 
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t£X 



(44) 



respectively, where for the sake of simplicity we assumed that the observation vector x is a continuous random 
vector. Extension to any kind of random variable is straightforward. In similar, the MSE bounds in (40) and 
(41) can be rewritten as 



/ / 



.<p + lh,ip)\ fo\M*) dp 
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and 
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(46) 
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respectively. Thus, the proposed bounds on the outage error probability and on the MSE depend on the LR 
functions. 

This result is consistent with previous literature. For example, in [3] it is shown that some well known 
non-Bayesian MSE lower bounds of unbiased estimators are integral transforms of the likelihood-ratio (LR) 
function. In addition, it is well known that the WWLB [20] is a nonlinear function of the LR function and the 
Bayesian Cramer-Rao bound and Bobrovsky-Zakai bounds are special cases of this bound. Furthermore, the 
ZZLB is a function of the likelihood ratio test of binary hypothesis testing, and therefore, it is also a function 
of the LR. 

VII. Examples 

In this section, the performance of the proposed lower bounds derived in this paper, are evaluated via 
examples. 

A. Example 1 

In this example, the single coefficient bound and the tightest bound from (21) and (33), respectively, are 
derived for the following linear Gaussian model: 



x = 6 + n, 9 ~N ' (fig, of) , n~A/"(0,CT^) 



(47) 



where 6 and n are statistically independent and the notation N(n,o 2 ) represents a normal density function 
with mean /j, and variance o 2 . The conditional distribution of the parameter 6 given the observation vector x 

(\ 2 2 

E[0|:c], Oq< ) where o\ = /, " 2 . It can be seen that in this case the conditional pdf is symmetric and 

unimodal and thus, the h-MAP estimator is identical to the MMSE estimator for any h > 0. Using (1), the 

minimum /i-outage error probability attained by the h-MAP (or MMSE) estimator, is 
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E[%]-| } 9 \ x 



dip 



1 — erf 



h 



2j2o 2 



(48) 



I*, 
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where erf (x) = —j= Jjf e * ctt is the error function. The single coefficient bound (21) with p — > 1 + is 



Pr 



> 



max < 0, 1 — /jpE 



> max <^ 0, B 



p 

,-1 

?|x 



>(1) 



r-P 



/r-'faWdy. 



p-1 

p 



max < 



0, 1-hp- 



2vra 



2 \ 2 P 

! |x 



P — 1 \ 2 P 



► , p > 1 . (49) 



It can be seen that the single coefficient bound with p — > 1+ is tight for h < 1. However, we obtain tighter 
single coefficient bounds, for example, using p = 1.5 and p = 5 for 1.5 < /i < 3 and h > 3, respectively. 
According to Theorem 3, for the symmetric and unimodal conditional pdf in this case, the tightest bound in 
the proposed class presented in (33) is identical to (48) and it is tighter than the single coefficient bound for 
all h and p. 



B. Example 2 



Consider the following parameter estimation problem with posterior pdf: 



/j* ^px rp (fix 

f$\x(v\x) = ^e'^u(ip) + 7^-e^(l - u(<p)) 



(50) 



where u(-) denotes the unit step function and x is an arbitrary positive random variable. It can be seen that 

this conditional pdf is unimodal but it is symmetric only if Ai = A2. 

Table I presents the MAP, MMSE, and /i-MAP estimators for this problem derived under the assumption 

that Ai < A2 and the corresponding probability of /i-outage error. The notation in this table: a x = ~ , 

TABLE I 

The estimators and probabilities of error for Example 2 



Estimator 


Probability of /i-outage error 


Om ap = 


IE 


— hx —hx 
g 2Aj^ _|_ g 2A 2 




VMMSE — 2x 


1-lE 


f min(a x J rh,0) — maxfa^ — h,0) iaiii.(a x —h,Q) — max.(a x -\- h .0) \ 
J — g 2A 2 _|_ g 2A! — g 2A 2 — g 2Ai | 




®h-MAP = dx\h) — 2 


IE 


e mm(0 A (/.)-^)^ + e -max(0,4W+Y)xr' 
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c x {h)= log^ + 



- g) ^fey^ and 



ifc x (/i)<0 

4(M = <i ^(/i) if 0<c x (/i) </i 

h if <<j(fo) > /i 

Using (33) , the proposed bound on the probability of /i-outage error is 



ii 2 



» Ao 



d x (h)i 



+ e A i 



(51) 



x = < 



The conditional pdf /6/i x ('l x ) i n (50) is unimodal and thus according to Theorem 3 it is identical to the 
minimum probability of error for each h attained by the /i-MAP estimator with h = h, presented in Table I. It 
can be seen that the h outage error probabilities of the h-MAP estimator with h = and the MAP estimator 
are equal. 

Consider the case of discrete distribution of x: 

1 with probability 0.5 

2 with probability 0.5 

and Ai == 1. The minimum /i-outage error probability and the proposed tightest bound for this distribution 

of x are presented in Fig. 1 as a function of h for distribution parameter A2 = 10. In addition, the outage 
error probabilities of the MMSE, MAP, and /i-MAP (h = 5) estimators are presented in this figure. It can be 
seen that for h — > the proposed bound approaches the MAP outage error probability. The MMSE estimator 
approaches the bound for h > 14. 

Fig. 2 shows the /i-outage error probability with h = 20 obtained by MMSE, MAP, and h-MAP (h = 20) 
estimators compared to the proposed bound versus j-. This figure shows that the lower bound on the h- 
outage error probability with h = 20 coincides with the performance of the corresponding h-MAP (h = 20) 
estimator. 



C. Example 3 



Consider the following observation model 



w 



(52) 
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A. 2 =10 





- minimum probability of h-outage error 


o 


proposed bound 


□ 


MMSE - Monte Carlo 




MMSE - theoretical 


V 


MAP - Monte Carlo 




MAP - theoretical 


+ 


h-MAP (h=5) - Monte Carlo 




- h-MAP (h=5) - theoretical 




Fig. 1. Probability of error versus h for A2 = 10 and unimodal non-symmetric exponential pdf. 



where the unknown parameter, 9, is distributed according to 



fo(<p) = c (u(f - 3) - u(f - 6) + u(f + 6) - u(f + 3)) , 

O 



it; is a zero- mean Gaussian random variable with known variance a 2 , and 6 and w are statistically independent. 
The /i-MAP estimator for this problem is 



dh-MAP = < 



(6 - |) sgn(x) \x\ > 6 - | or g(h) < \x\ < 1.5, 6 < h < 9 
(3 + %) sgn(x) |x|<3 + |,/i<6 



(53) 



h> 12 



otherwise 



where g(h) is the solution of 2erf I — -£= I = erf ( ^ I — erf I -J== I and sgn(-) it the sign function. The 
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0.9fa 

0.8 

0.7 



h=20 

minimum probability of h-outage error 

o proposed bound 

n MMSE - Monte Carlo 
MMSE - theoretical 

v MAP - Monte Carlo 
■ - - MAP - theoretical 

+ h-MAP (h=20) - Monte Carlo 




Fig. 2. Probability of ft-outage error for ft, = 20 as a function of j- for unimodal non-symmetric exponential pdf. 



corresponding minimum probability of /i-outage error is 



min Pr 

0(x) 



> 



± ( 3 - (Serf (j=) + 3erf 



+ 3erf 



i(3 + 3er/ (a< 
A -erf 



12-h I i „„f I h 



6erf 



e-g(h) 



+ 



/2^2 

'2<r 2 



+ 



e 2<r z 



(3+hy 



_ 9 _ 36 

e 20^ — e 2T 2 



+ A/^le 



2^~ 



(6-g(.h)Y 



2V2<r 2 







where b(h) = 6 — h — (6 — 2h)erf 



h 



< h< 3 
3< /i< 6 

6 < /i< 9 > ( 54 ) 

9 < h< 12 
/i > 12 

It should be noticed that the h-MAP estimator is not unique for 



,272^2, 

h > 3. In addition, there is no single estimator that attains the minimum probability of /i-outage error for 



IS 



every h. The MAP estimator is obtained by (53) in the limit h — > 0, that is 



6 map = < 



Qsign(x) \x\ > 6 

x 3 < |x| < 6 

3sign(x) \x\ < 3 



and the MMSE estimator in this case is 



Ommse = x + 



2cj 2 



(g+6T 

e 2<t 2 



(x+3r 

e a? 2 



(*-er 



e 2<T 



2" 



(*-3)' 



e 2<r 



T~ 



c(x) V IT 

where c(x) = er/ (^=) - er/ (-7=7) + erf (-7=7) - erf (4±=Y Using (33) and the "valley-filling" 
operator, the proposed bound on the outage error probability in this case is 



B 



(o) 



1 (UU\ U^f I i+ 3 



b(h) — herf 



'2<j< 



2 
12-/i 



9 



2V2^2 
h 



[I -erf 

{ 1 ~ erf \2V2^ 





0< h<3 

3<h<9 

9 < /i< 12 

h> 12 



(55) 



The Ziv-Zakai probability of outage error lower bound (defined in Appendix D) for this problem is 



i max (b(h)- h,3-3erf^^)) 0<h<3 



IzZLB(h) 



hi -erf 
V 1 -erf 



9 



' V2v / 2^2 
12-h I „„j- ( h 



2V2a 2 







3< h< 9 
9 < /i< 12 

/i> 12 . 



(56) 



The minimum /i-outage error probability, the proposed tightest bound, and the ZZLB on the probability 
of outage error are presented in Fig. 3 as a function of h for a 2 = 100. In addition, the outage error 
probabilities of the MMSE, MAP, and /i-MAP (h = 1.5) estimators are presented in this figure. It can be seen 
that the proposed bound approaches the minimum outage error probability for every h and for h < 3 there 
is a significant difference between the proposed bound and the outage error probability used in the ZZLB 



expression. 
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Fig. 3. Probability of error versus h for linear non-Gaussian model with er" = 100. 

VIII. Conclusion 

In this paper, new classes of lower bounds on the probability of outage error and on the MSE in Bayesian 
parameter estimation were presented. The tightest subclasses of lower bounds in the proposed classes have 
been derived and the tightest bounds in these subclasses are presented. For unimodal conditional pdf the 
tightest outage error probability lower bound provides the minimum attainable probability of outage error. For 
unimodal and symmetric conditional pdf the tightest proposed MSE bound provides the minimum attainable 
MSE. It is proved that the proposed MSE bound is always tighter than the well known ZZLB. The applicability 
of the proposed bounds was shown via examples. 

Appendix A. Proof of Theorem 1 
Sufficient condition: Let gh(x,8) be periodic in 9 with period h, for almost every x € x- I n particular, 



0+# 



g h (x, <p) dtp = c(x, h) , V0 



(57) 
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where c(x, h) is not a function of the estimator 9. Thus, using (7) and (9) 



E[K(x,0H(x,0)|]=E 



e+i 



g h (-x,(p)d(p 



E[c(x,/»)] 



(58) 



is independent of the estimator 9. 

Necessary condition: Let E [|u/i(x, 9)vt (x, 9)\] be independent of the estimator 9 and define the following 
family of estimators 

6U,a(x) = a^lxeA (59) 

where x is a random observation vector, the indicator function, l x eA> is defined as 

{a.A if x G A 
, a A €R , 
o if x g A c 

and A c is the complementary event of A Then, under the assumption that E [(-u^x, 8)vh (x, 9) |] is independent 
of the estimator 9, in particular, for each estimator ^ iQ (x) 



E[\u h (x,9)v h (x, 



E 



E 



E 



0A, a {*)+ 1 : 

0A,c«(x)-f 



ff/ l (x,(/?)d( / 9 



L x eA c 



^(x,y?)d(^ 



5/ l (x,(^)d</? 



+ E 



+ E 



LxeA 



■«A+f 

IxeA / g h (x,<p)dip 



a A - 



oc A + L . 



a A - 



g h (x,<p)d<p- / g h (x,(p)d<p 



(60) 



is independent of A or 04. Thus, E 



LxeA 



"OA + 



L _* 5/i (x, 92) dv9 - J_\ h (x, <£>) dy> 



is identical for every A 



and 0^4. In particular, by setting A = where is the empty set, one obtains 



E 

E 



L xe0 



L _l S7i(x, <p) dip - f\ g h {x, p) dip 



«0 



!x 6 a [L A _h gh(*,<p)d<p- f\9h(x,<p)d<p 



a a 



0, and therefore 
= for every A. Accordingly, 



a A - 



O.A- 



g h (x,p)dip- g h (x,tp)d(p = 0, a.e. x G x, V«a G 



(61) 



Equation (61) indicates that under the assumption that /0i x (y>|x) > 0, Vx G X; 9? G M, the term 



E 



L_fc 5 , h(x,(^)d^ 



is independent of the estimator 9 only if ^(x, 0) is a periodic function w.r.t. with 



period /i for a.e. x G X- 
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Appendix B. Eq. (25) yields a global maximum of the bound in (24) 

In this appendix, it is shown that the stationary point of (24) w.r.t. the Fourier coefficients {afc(x, h)} 
is a global maximum, that is, it will be shown that the Hessian matrix of (24) w.r.t. the coefficients 
{afc(x, /i)}fcgz,fc^o is a negative-definite matrix at the point {a^ (x, /i)}fcez,fc^o which satisfies (25). The deriva- 
tion in this appendix is carried out under the assumption that x is a continuous random vector. Extension to 
any random vector x satisfying ||x||2 < oo is straightforward. For example, if x is a discrete random vector 
with the probabilities P(x = Xj) = Pi, i = 0, 1, . . ., the pdf / x (xo) is replaced by po along the proof. 

Under the assumption that the integration and derivatives can be reordered, the derivatives of (24) w.r.t. 
{a TO (xo, h)} , Vm G Z, m / 0, xo G X are 

d %P 1,1 I, ,„, J ff ^ , ,,,:- ^ - 



ao m (x ,/ i ) *Mfo. *)/*(*>) (/ f JT a fc (x»e^H P / e pMx)dJ x 

\1R / 

/ \ -2- 

r / °° \ !-p p 

V ^ ■ 2,7vk \ — — ■ 27TT71 

J^ a k ( Xo ,h)e l —v\ ffc 1 ( V |x )e l — * dtp, (62) 



*>fc=— oo 



1 



/' 



and by equating (62) to zero (note that J ( YlT=-oo a fc( x 5 h)e l h f J p /JL 1 (y|x) dy > according to Holder's 

inequality properties), one obtains 

p 
/ £ a^Cxo.^e*^ /^(HxoJe^d^O, V?n G Z, m + 0, x G X • (63) 

1/18 \fc=-oo / 

The second order derivatives are obtained as follows 

HL,i 



<9a m (xo,/i)<9a^(x ,/i) 
^J-^^4(xo,fe)/ x (xo) I y I J2 a*(x,fc)6*^J /^(^|x)d V 



1 1 I I r / ^L .,_!. \ 1 -p 



1 
\ i 

.■ 2trfc , 



P 

X 

'" \ ■ I " P|X v • 1 / 'I 

&! = — OO 



p / \ p 



/ £ a fc (xo»e^ /^MxoK^^d^/ £ a*(xd,ft)e*^ /^|x )e-^ dy> 

-1 



1 



/ 1 ia|(x ,/ J )/ x (x )( /( ]T a fc (x,/i)e i2 ^) '/A^d*) 



7" r "oi. A U!'VJxv j, 'u; 1 / 1 / _ , "-fcv A )"-;° 1 jgi 



fe=— 00 

2p-l 
1-P p 



J [ E OfcCxo,^^) /^(^Ixo)^ 2 ^^^^ (64) 



*>fc=— 00 
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for all m,l G Z, m, I ^ 0, xo £ x, where H is the Hessian matrix. By substituting (63) in (64), the Hessian 
matrix at {a k (xo,/i)} is 

H ({4( x 0)^)f) =-ci(*o,h) c 2 (xo,(p,h)[F] m> i(cp)d(p, (65) 

L \ l J / J m,l J K 



where 

-i 

i -i I r f °° \ T ~ p ~ p 

ci(x ,/i) = /tPa£(x ,/i)— y/ x (x ) / I Y^ a fe (x,/i)e J ^~^ j J^" 1 (<^|x) d^ 

\R \k=-oo / 

2p-l 

(OO \ l_p p 

Y^ 4 O) ( x 0,^)e J ^^J /^'(c^lxo), 

k=— oo / 

■ 27r(m — i) 

and [F] m z(<£>) = e* fc v is the m,/th Fourier coefficient for any period h and for all <p G R. The periodic 
function, X^fcL-oo a L ( x Cb h)e t ~h~ ip , is positive for almost all <p G R and xo € X an d in particular ao( x o> fr) > 
(according to (10) {cjfc(x, /i)} should be chosen such that ]T]fc^=-oc °fc( x 0; h)e l ~h~ ip is a positive function) and 
thus ci(xo, /i) > and C2(xo, y>, /t) > for all h > 0, p > and for almost all </? € R and xo G x- For given 
ip G R, the infinite matrix of the Fourier coefficients, F(y?), is known to be positive-definite. Therefore, H is 
an infinite positive-definite matrix and (63) yields a maximum of (24). 

Appendix C. Unknown parameter with bounded support 

The derivations in Sections III and IV are carried out under the assumption that the unknown parameter 
is unbounded with fg\ x (0\x-) > 0, V# G R for almost all x G X- I n this appendix, it is shown that the derived 
bounds are suitable also for parameters with any support. The support of the unknown parameter 9 for given 
observation vector, x, is defined as 



Se\ x = W ■ /e|x(^l x ) > 0, tp G R} 



where A is the closure of a set A. 
Let define the function 



if G S elx 

e(x,0) = < , (66) 



„-> 



S^tse-^ if^V 



2.S 



where 5 > 0, a S> 1. Using this definition, it can be stated that 



E 



and 



MM)|i-p 



E 



/e|x(¥'|x)|%(x,yj)| 1 -p dy? 



E 



(/e|x(^|x) + e(x,v?)) ^(x,^)! 1 -? dip 



(67) 



Ep 



/e|x(¥'|x)|v/ 1 (x,^)| dv? 



< Ep 



(/e|x(¥>M + e(x, ¥>)) [« h (x, 93) | d<^ 



, P > 1 • (68) 



Using (8), (67), and (68), one obtains the following lower bound on the outage error probability 



Pr 



> 1-Ei 



> 



e+i 



(/0|xM x ) + e ( x >^))K(x,¥>)| dip 



p-1 
E v 



(/fl|x(y|x) + e(x, y>)) |u/,(x,e)|i-J> 



(69) 



for all <5 > 0, j> > 1. In similar to Theorem 1, define the function 



A 



g h (x,9) = (f e]x (e\x) + e(x,e))\v h (x,9)\ 



(70) 



A valid bound which is independent on 9 is obtained iff ^(x, 0) is periodic in with period /i, for a.e. x € % 
and G R and the periodic extension ^(x, 9) can be represented using Fourier series [18]: 



9h 



(x,0) = Yj h(x,h)e i2 l k(> , a.e. xGx- 



(71) 



k=— oo 



The derivation in Section III is valid for bounded support where ^(x, 9) is replaced by <?/}(x, 9). The general 
class of lower bounds on the probability of error in the bounded support parameter estimation problem is 



p-i 



Bu = l-h*Ep[bo(x,h)]E~ 



i 
i-p 



Y, 6*(x, hy^t* (fe\Mx) + e(x, y>)) ~ d^ 



\k=—oo 



, (72) 



for all p > 1 and h > 0. The tightest subclass of bounds in this class is given by Pr 
p > 1, where 



M 0) =l-E 

2 '^ 



p-1 



/ I 13 (/e|x(y + ^|x)+£(x,^ + Z/i))^ I d^ 



\l=— oo 



^ /A \ 6(°) 



(73) 



Using (66) and taking the limit 5 — >■ in (72) and (73), yields the bounds 



B ^p 



p-i 



1-hvEv [6 (x,/t)]E p 



^ 6fc(x,/i)e 



» 2?rfe „ 



1 
1-p 



.(^Ix))"" 1 d(^ 



v,fc= — OO 



, P>1, (74) 
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and 



B 



(o) 



1-E 



V 



Y, fiZHv + lhl*)] cV 



\l=— oo 



p > 1 



(75) 



respectively. In particular, for p->l + , the bound in (75) becomes 

r rh 



D 



(o) 



1-E 



max{/ e | x (v9 + //i|x)} d^ 

Q l&£ 



(76) 



which is the tightest bound on the outage error probability in the proposed class of lower bounds. Accordingly, 
the proposed bounds can be applied to any parameter estimation problem with unknown continuous random 
variable. 



Appendix D. The proposed MSE bound is tighter than the ZZLB 

In this appendix, it is analytically shown that the proposed MSE bound in (41) is always tighter than the 
ZZLB. For the sake of simplicity, we will assume that x is continuous random variable. Extension to any 
random vector x satisfying ||x||2 < oo is straightforward. The MSE lower bound from (41) can be rewritten 



as 



oo oo n 

Ci = 2 / E / ^|x(vl x ) dl P ~ / max/e|x(^ + lh\x) dip 



hdh. 



(77) 



L-oo 

Using the convergence condition in (26), the first integral in (77) can be divided into infinite sum of integrals, 
where each integral is over a single period and (77) can be rewritten as 



C^ = \ I 



" oo 

/ mm Yl Mxfo + kh \^ dip 



k= — oo 



hdh. 



(78) 



The ZZLB (without the valley-filling function) is [7] 



oo / oo 



1 



Bzzlb = r 



(Mv) + /^ + /i))f m mfe^ + ^)#| hdh (79) 

Voo 

where /#(■) is the a-priori pdf of 6 and P m i n ((p,cp + h) is the minimum probability of error for the following 



detection problem: 



#0 : /x|H ( x ) =fx\e(*\<p) 
Hi ■■ /xii/ifc) = fx\e(x\<P + h) 



(80) 



2", 



where / x |i?j(")> i = 1, 2 is the pdf of the observation vector x under each hypothesis, and the prior probabilities 

M<p) 



arc 



P(Ho) 



-, P{H X ) = 1 - P(H ) . 



(81) 



It is well known that the minimum probability of error is obtained by the MAP criterion and for binary 
hypothesis testing it is given by [21] 



Pmin( ( P,f + h) = 1 -E 



max PiHAx) 

i=l,2 



E 



min P(HAx) 

i=l,2 



(82) 



By substituting (80) - (81) in (82), the minimum probability of error obtained by the MAP detector is 



Pmin(tp,<p + h) = E 



mm 



fx\o(x\v)M<p) /x|e(x|y + h)f e (ip + h) 



(fob) + fo(<P + h)) ' /x(x) (feb) + fob + h)) 



(83) 



Note that since x is the only random variable in (83), the expectation is performed w.r.t. x. By substituting 
(83) in (79), one obtains 



Bzzlb 



E 



-co 

CO CO 



mm 



f*\e{*\v)feb) /x|e(x|v? + h)feb + h) 



/x(x) ' / X (X) 

- / / E[wm(f e \ x ((p\x),f e \ x (<p + h\x))] d(phdh. 



dphdh 



(84) 



-co 



Since f E [min (/g| x (v|x), fe\ x (<p + /i|x))] dip < f E [/^i x (y>|x)j dip = 1, the inner integral in the r.h.s. of 

— CO —CO 

(84) converges (i.e. the ZZLB does not need a convergence condition on the pdf). Therefore, it is possible 
to change the order of integration w.r.t. x and tp, and the integral can be divided into an infinite sum of 
integrals. Thus, 

CO 

1 



Bzzlb = „ / E 
o 

CO 





min(/e|x(y|x),/e| x (^ + /j|x)) dip 



hdh 



00 r(l+l)h 

Y^ J min (fo\ x (<p\x), fe\ x (<p + h\x)) dtp 



i = — CO 



III 



hdh 



CO 

o Lo 



Yl min(/ | x (v? + //i|x),/ e | x ((/? + (/ + l)/i|x)) dp 



l=—oo 



hdh 



(85) 
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Thus, the ZZLB can be written as Bzzlb = \ J IzzLB(h)hdh where 



7 

IzzLB(h) = V < / E[min(/ e | x (</>|x),/0| x (^ + h|x))] dip 



is referred in this paper as the Ziv-Zakai outage error probability lower bound. 



M-l 



M- 2 



In [19] it is shown that min Y2 a « — Y2 mm (°n> On+i) for any M non- negative numbers {a n } n=Q . In a 



n=0,n^£k 



n=0 



similar manner, it can be shown that for an infinite countable set of non-negative numbers {a n } ne z satisfying 



lim a n = and Yl o-n < oo, 

n->±oo „=_oo 



mm 

fcez 



^ a n > ^J min(a n ,a n+ i) . 



n=— oo, n^fc ra=— oo 

In particular, under the convergence condition in (26) 



mm ^ /0|xO + lh \*) > ^2 min CfyxOP + Z/l l x )> /e|x(<£ + (^ + l)/i|x)) 



fcez 



l=—oo,l=£k 



l=—oo 



and thus 



E 



mm J] /9| x (yj + lh\x) 



fee 



/ = — oo 



> E 



X] min (/e|x(v + lh \*)> /e|x(y + + l)^|x)) 



i=— oo 



Therefore, from (78), (84) and (88), one concludes that 



C x > Bzzlb 



(87) 



(89) 



Thus, the proposed lower bound in (41) is always tighter than the ZZLB in (79). Applying the valley-filling 
operator on both sides of (89) does not change this result. 

Appendix E. The tightness of the bound for unimodal pdf 

In this appendix it is shown that if the conditional pdf /gi x (-|x) is a unimodal function, the outage error 
probability bound in (33) coincides with the minimum probability of outage error in (1) for every h > 0. 
Assume that /ei| x ( - |x) is unimodal with maximum point 9map{~x) = #o( x )- The bound in (33) involves the 
computation of 



E 



max{/ e | x (<^ + //i|x)} dip 

Q l^£ 



E 



| max UeJo (x) + hll- 



h 



dip 



(90) 
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Since the conditional pdf, fm x (-|x), is unimodal with maximum at #o(x), then 



h 



max 



{/e|x (0 o (x) + /ia x M|x) , f e \ x (^o(x) + h (a x (^) + 1) |x) } 



(91) 



for all x and ip G (0, h) where a x (<p) 



6g(x)-<p 

h 



flp(x)-y 



- and ["J is the floor function. Note that the function 
o x (-) is continuous almost everywhere for ip € (0, h) and x G x and i n the continuous region its derivative is 



da x (y) _ 1 



d<p 



j-. By substituting (91) in (90) and changing variables to ip' = a x (tp), one obtains 



E 



uo 



max{/ e | x ( v ? + //i|x)} dtp 



E 
E 



max 



\fe\x (^o(x) + ha x {tp) |xj , / | x ^o(x) + h (a x (<p) + 1) |xj | d<? 



ft max |/ e | x (8 (x) + hip'\x) , f \ x [0 (x) + /V + h\xj j d<// 



(92) 



Changing variables to tp = hip' + #o( x ) and decomposing the integral to two regions, results in 



E 



max {f g \ x (<p + lh\x)} dtp 



E 



00 (x) 



6»o(x)-/i 



max{/ | x (</3 + /i|x),/ | x ((/3|x)} d</> 



E 



E 



0h(x)- 



e (x)-h 

'h(x)+- h 



max{/ e | x (( / 3 / + / i |x),/ e | x ((^ / |x)} d</?' 



0o (x) 



oiax{/ | x (^|x),/ 9 | x (^-/t|x)} dtp 



+ E 
+ E 



0o (x) 

0fc(x)-| 

00 (x) 

h(x)-| 



max{/ | x (93|x),/ e | x (v? + /itx)} d<£ 
max{/ e | x (v9|x),/ e | x (v? + /i|x)} dy> 



(93) 



where the /i-MAP estimator, #h(x), is defined in (2) which maximizes the area under the curve of the condi- 
tional pdf for given length h. Thus, in the unimodal case, the /i-MAP estimator is the unique estimator that 
satisfies the equation / | x (0/i(x) — ^ l x ) = fo\x (^( x ) + 2l x ) ^ or a ^ ^ > 0- Using the unimodal property, the 

conditional pdf satisfies 

( 

fg\ x (<p - h\*) < fo\*((p\x) (x) <tp< § h (x) + \ 
fo\x(<P + %) < fo\x(<P\x) ^( X ) - I < f < ^0(x) 

for all x S x and (93) can be rewritten as 



E 



■ rh 

j max{/ e | x (v? + //i|x)} dtp 
Jo <•€■*> 



E 



E 



»h(x)+ 



00 (x) 
0h(x)+| 

0h(x)-| 



/e|x(vl x ) d^ 



+ E 



00 (x) 

0h(x)-| 



/e|x(^|x) d</3 



/eixC^lx) dy 



(94) 



2.S 



By substituting (94) in (33), the proposed bound on the /i-outage error probability in the unimodal case is 

r4(x)+| 



^ =1-E 

12 - 1 



h (x)-$ 



(95) 



which is identical to the minimum probability of /i-outagc error in (1) obtained by the h-MAP estimator. 
Thus, the proposed outage error probability bound in (33) coincides with the minimum probability of outage 
error in (1) for every h > for unimodal conditional pdf. 
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